1. Field of the Invention
The present invention relates to a photographing apparatus such as a digital still camera and a photographing method using a photographing device in an interlacing transfer system where upon transferring electric charges stored in a photographing device the total pixels are divided into a plurality of fields, and a color signal comprising at least RGB or YeCyMgG is involved in transfer data in each field.
Incidentally, the present invention further relates to a dynamical defect involved in a digital still camera constructed with a solid pick-up device such as a CCD and furthermore relates to an image information processing apparatus.
2. Description of the Prior Art
In a digital still camera where CCDs are employed as pick-up devices, a cell constituting each pixel of the CCD generates electric charges in response to the rate of received light. The amount of electric charges in each cell are read out as the value of pixels, which value of pixels is outputted subsequently from the CCD as image data for a photograph per 1 frame.
In a photographing apparatus used for a prior art digital camera, there are known transfer of electric charges from a photographing device in conformity with an interlacing system where reading is performed twice or a progressive system where readout is performed once.
These systems have merits and demerits. For example, a digital camera disclosed in Japanese Laid-Open Patent Publication No. 2001-285688 operates in both systems of interlace transfer and progressive transfer. The interlacing transfer system in the above disclosure provides a high quality image by making use of a single shot mode with a shatter although long processing time is required, while the progressive transfer system ensures high speed by making use of a continuous photographing mode where a signal of a one pixel train is transferred to a plurality of pixel trains without the use of a shutter, although smear might happen.
Further, in accordance with a video camera, a measurement of light, and a detection method for focusing disclosed in Japanese Laid-Open Patent Publication No. Hei 6-54250, preliminary photographing is performed to adjust the rate of exposure to light by making use of the divided one field in an interlace transfer system, and then to adjust a focused position.
However, in a photographing apparatus using pick-up devices in an interlacing transfer system with such a construction, upon electric charges stored in a pick-up device being transferred, the signal is classified into R, Gr signals and B, Gb signals in the case of color filters in a primary color system or into Ye, Cy signals and Mg, G signals in the case of color filters in a complementary color system, which are transferred in two fields of first and second fields as illustrated in FIG. 7. In this transfer system, data required for an image processing for the RGB signal or the YeCyMgG signal can not be obtained until the transfer of the two field fractions of the first and second fields to result in much time for the image processing. Further, even in preparations of an image for checking photographing and of a thumbnail image similar processing are performed while requiring some time.
In general, a CCD for use in a photographing apparatus includes as illustrated in FIG. 38 a pixel S arranged in a matrix, a vertical transfer register VR provided for each vertical line along a horizontal one side of each pixel S, and a horizontal transfer register HR provided on the side of a final end of each vertical transfer register VR. In this CCD, any one among three primary filters of R (red), G (green), and B (blue) is disposed on each pixel S. The pixel S on which the R filter is disposed converts the light of R and outputs an R signal, and a pixel S on which a G filter is disposed converts the light of B and outputs a G signal, and a pixel S on which a B filter is disposed converts the light of B and outputs a B signal.
In the foregoing three primary color filters, regions from which the respective signals are outputted are arranged horizontally in the order of R, G, R, G . . . or G, B, G, B, and a horizontal line (hereinafter referred to as an RG line) of the arrangement of the color filters of R, G, R, G . . . and a horizontal line (hereinafter referred to as a GB line) of the arrangement of the color filters of G, B, G, B . . . are alternately provided. It is noticed that in order to discriminate the G signal from the RG line and the G signal from the GB line the G signal from the RG line is described as a Gr signal and the G signal from the GB line as a Gb signal in the figure.
In the prior art interlacing transfer method, upon transferring electric charges stored in the CCD constructed as described above the regions are divided into the field constructed from the GB line (refer to FIG. 4(a).) and the field constructed from the RG line (refer to FIG. 4(b).) for transfer with two fields.
In the prior art transfer system, however, the field constructed from the GB line contains only the Gb signal and the B signal, while the field constructed from the RG line contains only the R signal and the Gr signal. More specifically, since all kinds of RGB signals are transferred only after the transfer of the two fields is finished, data required for the image processing of RGB cannot be obtained until the transfer of all fields is finished.
It is accordingly impossible to prepare image characteristic data in the whole of an image up to the completion of the transfer of the image, and hence it is impossible to start image processing thereafter. There is therefore problems that much time is required until the image processing is completed and too much time is required until the next photographing since the photographing.
Further, in such a digital camera which employs a CCD as a photographing device a cell of the CCD constructing each pixel generates electric charges in response to the quantity of received light. The quantity of the electric charges in each cell is read as the value of pixels, which value is in turn outputted in succession from the CCD as image data for a photographic image of 1 frame. The digital still camera is improved in its miniaturization and in high resolution. This however causes a severe problem of a pixel defect occurring on part of each cell constructing a pixel as the number of such pixels is increased.
The pixel defect of this kind includes defects called a white defect, a white defect, an in-bright white defect, and a temperature white defect. The black defect originates from dust and the like deposited on a light incident surface of a pixel, and the in-bright white defect originates a defect on a color filter or a micro-lens disposed on the light incident surface.
Any of these black defect and in-bright white defect is a static defect happening irrespective of an operation temperature environment of a CCD, which is being improved as a technique of the manufacture of a semiconductor for use in the manufacturing process of a CCD.
In contrast, the temperature white defect is also called a dark current defect, which is a dynamical one influenced by a change in a dark current and accumulation of electric charges, i.e., a change in the temperature of a CCD and a change in exposure time to light.
For correcting the temperature white defect being a dynamical defect, Japanese Laid-Open Patent Publication No. 2000-10192 or No. 2000-224487 proposes a technique wherein an address of a pixel where the temperature white defect of a CCD occurs is previously stored in a memory, and the value of a defective pixel is corrected on the basis of an address stored in the memory upon photographing with a digital still camera.
However, the temperature white defect changes owing to temperature and exposure time to light, so that for obtaining the address of a pixel on which the temperature white defect occurs it is necessary to take temperature and exposure time to light as parameters, and investigate on which pixel a temperature white defect of a level causing a problem under the conditions of those parameters, and prepare an address table thereof.
Such an investigation is required for every combination of the parameters, so that much labor is required for the preparation of an address table for the defective pixels, and hence it is not practical. Further, even if all address data about the temperature white defect can be prepared with a linear interpolation method from part of address data, there are required a temperature sensor for measuring operation temperature being one of the parameters and a memory for storing an address table in which addresses of defective pixels are listed. This causes a difficulty that the cost of the camera is increased.
A photographing apparatus is known as an electronic still camera in which electric charges of all pixels accumulated in response to an image focused with imaging means are transferred to a photographing device equipped with color filters for color decomposition, and predetermined image processing is executed to form an image, and which is being assembled as part of functions of a portable information terminal such as a cellular phone and a computer apparatus.
Further, for the photographing apparatus, there is being generalized “a photographing device in an interlacing transfer system in which upon electric charges of all stored pixels being transferred data transfer of the electric charges is executed, divided into a plurality of fields” to deal with an increase of the number of pixels and a requirement of making the device compact.
For example, when in a color electronic still camera for use in a photographic device in such an interlacing transfer system color filters for color decomposition are of a primary color system of red (R), green (G), and blue (B), information involved in one field is red (R)•green (G) information or blue (B)•green (G) information, and hence necessary information for preparing a color image is not obtained only with the one field.
In prior art, in such an electronic still camera there is a problem that until the transfer of all fields is finished, in other words until transfer of all electric charges accumulated on a photographic device is finished, no image processing is executed, for example “the time required for displaying a conformation image of photographing” is increased.
Further, in the aforementioned electronic still camera information of all electric charges is an object of the image processing, so that the image processing time requires a predetermined time irrespective of set resolution and a compression rate, which causes a problem of “inconvenience” where even when an object originally requiring less image information such as “images for low resolution and for high compression rate”, much time is required until the next photographing is enabled.
It is, therefore, an object of the present invention to provide a photographing device and a photographing method capable of solving the aforementioned problems.
To accomplish the above object, according to the present invention a photographing apparatus of interlace transferring type is provided. In one embodiment, it comprises:
a photographing device which carries out transfer of electrification of all pixels stored in the photographing device by dividing into a plurality of fields when transferring the electrification and which has a plurality of color filters and which includes a color signal of at least RGB or YeCyMgG in said transfer data of each field for transferring said electrification; extraction means for extracting characteristic data of image from transferred data before processing for image is started;
generating means for generating control value carrying out correction of image based on said extracted characteristic data; and
photographing processing means for processing image by use of control value formed by said characteristic data.
In the other embodiment, a photographing apparatus of interlace transferring type comprises:
a photographing device which carries out transfer of electrification of all pixels stored in the photographing device by dividing into a plurality of fields when transferring the electrification and which has a plurality of color filters and which includes a color signal of at least RGB or YeCyMgG in said transfer data of each field for transferring said electrification;
extraction means for extracting characteristic data of image from transferred data before processing for image is started; generating means for generating control value carrying out correction of image based on said extracted characteristic data;
photographing processing means for processing image by use of control value formed by said characteristic data; and
selection means for selecting either a first mode for processing and recording the transfer data of all pixels accumulated in said photographing device or a second mode for processing and recording transfer data of pixels less than said all pixels, wherein in said second mode selected by said selection means, later transferred data is processed and recorded by the control value generated by said characteristic data extracted from the previously transferred data with division of the plurality of fields.
The color filter provided in the photographing device is composed of original color of RGB.
The color filter provided in the photographing device is composed of complementary color of YeCyMgG.
The apparatus has an interlace (or interlacing) transfer in which transfers of 3 times carried out by dividing the plurality of fields are achieved and each field is thinned perpendicularly into ⅓.
The characteristic data of image is color distribution for control of white balance and control value for white balance is generated based on said characteristic. The characteristic data of image is also data in which edge component within a screen is extracted and a control value for enhancing the edge is generated based on the characteristic data. The characteristic data of image is data in which a color information within a screen is extracted and a control value of color converting coefficient is generated based on said characteristic data. The characteristic data of image is data in which distribution of brightness within a screen is extracted and a control value of contrast correction is generated based on said characteristic data.
The photographing apparatus generates processes image of digitalization and said characteristic data of image is data in which distribution of brightness within a screen is extracted and threshold of digitalized processing based on said characteristic data is generated as a control value.
According to the present invention, an photographing method of interlace transferring type is provided. The method comprises the steps of:
dividing data transfer of electrification of all pixels stored in an photographing device of interlace transferring type into a plurality of fields when transferring the electrification; providing a plurality of color filters in said photographing device; receiving transfer data from said image element which transfers data including a color signal of at least RGB or YeCyMgG to each field; extracting characteristic data of image from the transferred data; and generating a control value of effecting correction of image based on the extracted characteristic data to make image processing using said control value.
The present invention provides an photographing method of interlace transferring type comprising the steps of:
dividing data transfer of electrification of all pixels stored in an photographing device of interlace transferring type into a plurality of fields when transferring the electrification; providing a plurality of color filters in said photographing device;
receiving transfer data from said image element which transfers data including a color signal of at least RGB or YeCyMgG to each field;
extracting characteristic data of image from the transferred data;
generating a control value of effecting correction of image based on the extracted characteristic data to make image processing using said control value; and
processing and recording later transferred data by the control value generated by said characteristic data extracted from the previously transferred data with the division in the plurality of fields.
Transfers of 3 times of electrification are carried out by dividing into the plurality of fields and thinning perpendicularly each field into ⅓.
The photographing apparatus includes a solid photographing device which has a plurality of pixels compartmentalized in every predetermined regions and in which a pixel for detecting a pixel value of each color of three original colors through a color filter of said plurality of pixels is arranged in said regions. The photographing device is determined so that a defecting pixel for generating a white scratch of temperature which changes according to temperature is one or less in each region, each pixel value of all the pixels in the solid photographing device being read out with division of fields of three or more to form an image of one frame based on the read out pixel value.
The apparatus comprises:
means for obtaining a difference about the pixel value among the pixels at arranged positions corresponding to first and second fields detected through color filters of the mutually same color arrangement every said regions; and
means for judging that when a difference between two pixels exceed a predetermined threshold, one pixel of the two is the defecting pixel and for amending a pixel value of the one pixel based on a pixel value of the other pixel of the two.
The pixel value of said one pixel is rewritten by the pixel value of the other pixel.
The apparatus further comprises means for obtaining a difference between a predicated value predicated by a pixel value of each pixel of a third field from each pixel of the first and second fields and a pixel value obtained from each pixel of the third field, and means for judging that when the difference obtained by the said means for obtaining the difference exceeds said predetermined threshold, the pixel of the third field is said defective pixel and for rewriting the pixel value of the defective pixel by said predicated value.
When a photographing mode of indicating elimination of the number of pixels is selected, a field having a less defective pixel is selected about all three original colors, and an image of one frame is formed from a pixel value of pixel of the field. The photographing mode is a continuous taking-out mode.
A correcting device for defect of ordinary temperature is provided at prior to said correcting device for white defect of temperature. The correcting device for defect of ordinary temperature amending a defect of ordinary temperature which does not depend on temperature of said pixel previously to effect amendment of the pixel value of the defective pixel by said correcting device for white defect.
Further, a photographing method is provided, in one embodiment, it comprises the steps of:
preparing a solid photographing device which has a plurality of pixels compartmentalized in every predetermined regions and in which a pixel for detecting a pixel value of each color of three original colors through a color filter of said plurality of pixels is arranged in said regions, said photographing device being determined so that a defecting pixel for generating a white scratch of temperature which changes according to temperature is one or less in each region, each pixel value of all the pixels in the solid photographing device being read out with division of fields of three or more to form an image of one frame based on the read out pixel value;
obtaining a difference about the pixel value among the pixels at arranged positions corresponding to first and second fields detected through color filters of the mutually same color arrangement every said regions; and
judging that when a difference between two pixels exceed a predetermined threshold, one pixel of the two is the defecting pixel and for amending a pixel value of the one pixel based on a pixel value of the other pixel of the two.
In the other embodiment, a photographing method comprises the steps of:
dividing data transfer of electrification of all pixels stored corresponding to an image focused on an photographing device having a color filter for resolving color by means of a focused means into M(≧3) fields; and
carrying out image processing including at least YUV conversion by use of transfer data of m (<M) fields in which all color signals and necessary number of pixel are get.
In one embodiment, number of fields: M is odd number of 3 or more, m=1.
In the other embodiment, number of fields: M is even number of 4 or more, m=2.
When the transfer data of m field is get, image processing including at least YUV conversion is immediately carried out.
An image for confirming photographing is made by image processing including YUV conversion.
A thumbnail image is made by image processing including YUV conversion.
Condition of an image to be formed is set and image processing is carried out by use of transfer data of n (m≦n≦M) field to make said image to be formed.
Condition of an image to be formed is resolution of image and n is number of field in which number of pixel necessary to make an image of the set resolution is get.
As an example, M=3, an image of low resolution is prepared with respect to number of field: n=1, an image of middle resolution is prepared with respect to number of field: n=2, and an image of high resolution is prepared with respect to number of field: n=M.
In one example, M=4, an image of low resolution is prepared with respect to n=2, images of middle and/or high resolutions are prepared with respect to number of field: n=M.
In one example, M=6, an image of high resolution is prepared with respect to n=M, an image of low resolution is prepared with respect to number of field: n=2 or an image of middle resolution or low resolution is prepared with respect to number of field: n=4.
Condition of an image to be formed is compressibility of image, and n is number of field in which number of pixel necessary to make the image of the set compressibility is get.
In one example, M=3, processing of compressing the image is carried out on format of JPEG with respect to number of field: n=1.
In the other example, M=4 or 6, processing of compressing the image is carried out on format of JPEG with respect to number of field: n=2.
Further, an photographing apparatus is provided comprising:
a photographing device having a color filter for resolving color:
photographing means for focusing an image to be photographed on said photographing device;
a photographing processing part for converting to digital signal electrification stored in said photographing device and transferred; and
means for carrying out image processing including at least YUV conversion by use of transfer data transferred through said imaging processing part, wherein said photographing device is interlace transferring type and divides data transfer of electrification of stored all pixels into M(≧3) fields.
The color filter of the photographing device is composed of original colors of red (R), green (G) and blue (B).
The color filter of the photographing device is composed of complementary color of yellow (Y), cyan (C), magenta (M) and green (G).
In one example, M is 3 in the photographing device of interlace transferring type, and said apparatus embodies said photographing method as recited in claim 29 or 34.
In one example, M is 4 or 6 in the photographing device of interlace transferring type, and said apparatus embodies said photographing method as recited in claim 30, 31 or 34.
The apparatus can embodies the photographing method and has a displaying part for displaying an image for confirming pick up and/or an image of thumbnail.
The apparatus has also a removable exterior memory for storing the formed image.
The apparatus has an image holding section for storing the formed image.
An information processing apparatus of image comprising said photographing apparatus is also provided.